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Abstract 

The students' lack of motivation is a usual problem. The students value more the obtention of the 
degree than the developing of competences and skills. In order to fight this, we developed a 
gamification's experience based on merits and leaderboards. The merits are linked to the attainment of 
skills and competences that students usually do not appreciate. During the year 2016, we are carrying 
out a pilot involving 950 students at the Graduate of Engineering in Computer Science at the 
Autonomous University of Barcelona (UAB). 
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1. Introduction 

Engineering Professors face a problem increasingly widespread: even when the 76% of the 
students consider they feel interest or vocation for their studies, many of them do not seem to be 
sufficiently motivated (Lopez, Alarcon, Rodriguez & Casado, 2014). This situation is accentuated 
in abstract subjects like algebra or calculus, because the students considered them to be 
uninteresting or even useless for their training. But the lack of motivation is not related only to 
the contents (Lee & Hammer, 2011), the goal of many students is to get a degree that allows 
them to enter the labor market (Smith-Robbins, 2011). For this reason, they do not adequately 
value the assimilation of knowledge, or the development of skills and competencies that they will 
need to develop their work. This situation also discourages teachers, leading to a vicious cycle 
that worsens the learning process and it is not easy to break. The most visible consequences of 
this situation are the low grades and the high dropout rate of these students. This 
communication presents a gamification experience (Deterding, Dixon, Khaled & Nacke, 2011) 
aimed at engineering students, with the objective of motivating them to improve their academic 
performance from the acquisition of knowledge, skills and abilities. This increase in the 
motivation of the students should involve, among other things, an increase of the attendance 
ratio and an improvement of their grades (Biggs, 2003). 


2. Gamification at the University 

Despite the increase of gamification as a trend in several areas, there are few examples of its 
usage under the scope of higher education. In Prieto Martin, Diaz Martin, Montserrat Sanz and 
Reyes (2014) a combination between gamification, flipped learning (Bergmann & Sams, 2012) 
and just-in-time-teaching achieved its goal of improving the performance of Biology students. In 
Gonzalez Gonzalez and Mora Carreno (2015), the authors present a gamification experience 
involving 100 students of one Computer Engineering's subject. This experience increased the 
attendance ratio and the motivation of the students, but it was applied only to a small part of the 
subject's agenda. In O'Donovan, Gain and Marais (2013), the authors transformed some of the 
subject's tasks into puzzles in order to engage the students to solve open problems and to trey 
alternative and original solutions. Similar experiences conducted with MSc students (Barata, 
Gama, Jorge & Gonsalves, 2013a/b) have been useful to increase their participation. These 
experiences have been also useful to discover some downsides of gamification, as the feeling of 
unfairness generated when rewarding quantity over quality, or the complains about the 
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competitive approach of the course (instead of a cooperative one). The delivery of badges has 
been also used to incentivize high school students to participate in afterschool activities (Davis, 
2015). 


3. Description of the experience 

The gamification experience revolves around the merit. A merit is an abstraction that recognizes 
that a student has demonstrated a certain level of competence in a particular skill. Each merit has 
an associated icon, a score (not all are equally important or difficult to obtain), a name and a 
description that identify and make them unique. For example, Figure 1 shows the merit "Jedi 
Master" associated with the subject "Information and Security", which is awarded to the students 
who pass the whole subject at their first try. Another example is the merit “good programmer”, 
related with the laboratory practices of the same subject, which is awarded to the students who 
deliver code that is not only functional, but well-structured and documented. 


• Merit: Jedi Master 

• Description: Master of masters 

• Requisites: Pass the whole subject at the first try 

• Subject: Information and Security 

• Value: 1000 

• Course: 2016/2017 

• Icon: 



Figure 1. Detail of one of the merits from the subject 
“Information and Security” 


Merits can be defined in advance and presented to the students at the beginning of the course, so 
that they know what to do to get them and can plan to do so, or they may be secret (only its 
existence is announced, but not how to obtain them) to motivate students who meet Bartle's 
"explorer" profile (Bartle, 1996), allowing them to play inferring or guessing what to do to obtain 
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them. Note that the gamification experience also offers motivational elements for the other 
profiles: the "killers" will want to beat their rivals and reach the top of the leaderboard; the 
"achievers" will want to get all the achievements, especially those that are difficult to obtain and 
are more rare or unique because few students obtain them; Finally, the gamification experience 
provides the "socializers" with an excuse to join the community or to discuss how to achieve this 
or that merit. 

The students enter more deeply into the gamification experience as they obtain merits. To 
represent this progression, we have designed a three-level scale. The first level, called 
"chairwarmer" identify the students who have not obtained any merit, the second level, "one of 
many" identifies students who have achieved between one and five merit, finally, from the 
obtention of the fifth merit, the students reach the "future engineer" level. From the fifth merit 
obtained, it is considered that students have already entered enough the gamification experience 
and will continue participating although they cannot increase their level. In addition, the fact that 
the maximum level is relatively affordable for everyone provides a target to the segment of 
students who does not considers itself able to reach the first places of the leaderboard. All these 
students can target reaching the third level to differentiate themselves from those colleagues who 
does not put the slightest interest. Note that this gamification experience uses all the main 
gamification mechanics (collections, points, leaderboards, levels and feedback) identified in 
Cortizo Perez, Carrero Garda, Monsalve Piqueras, Velasco Collado, Diaz del Dedo and Perez 
Martin (2011). 

The tool which drives the gamification experience is a web platform called TOP ENGINYERIA 
(http:// top.uab.cat/) . Through this platform, teachers can create and give merits to the students, and 
students can check the level achieved by each partner by consulting the merits' leaderboard. In 
this leaderboard, sorting is based on the scores of each student's merits, therefore, students are 
ordered according to their skills and abilities. It has a short feedback cycle, as may be updated 
whenever a new merit is given. This allows certain immediacy between performing an action (for 
example, deliver some optional exercises) and obtaining a reward (in this case, get the merit 
"practice makes perfection"). 
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Figure 2. Top 6 positions of the leaderboard, filtered by subject 


Figure 2 shows a screenshot of this leaderboard. This leaderboard is tied to a reward system that 
rewards students based on their position at a particular time of year. The rewards tend to take the 
form of extra-curricular activities that expand the training of students, but they are not evaluable. 
For example, a tour to the servers of the faculty with the "Networks" professor, only for the first 
25 students, or forwarding a job or paid internship offer to the first 15. This reward system 
provides an element of extrinsic motivation for that segment of students who does not obtain 
enough intrinsic motivation from competition and prestige for reaching and keeping the top of 
the leaderboard. However, it is important to note that it is not the central element of the 
experience, but only an additional incentive to that placed near the top. The experience is 
designed so obtaining these rewards should not be perceived as the students' ultimate goal (Deci 
& Ryan, 1985). 

The last element of the gamification web platform is the student profile. The profile consists of 
a web page that displays a visual summary of the student's participation in the gamification 
experience. The profile shows principally the position at the leaderboard, a list of all the merits 
obtained, and the level it has reached. On one hand, this profile is useful for every student to 
know exacdy what is its state in the gamification experience, how many merits needs to level up, 
which merits has already obtained and the value that they have, how is the student positioned in 
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comparison with his friends, etc. Furthermore, students have the option of making their profile 
public, so that anyone can see it. This option allows them to share it on social networks in order 
to obtain recognition, or, for example, use it to support their resume when they want to enter the 
labour market, or simply to request a grant or apply to a business' practices. In Figure 3, we 
provide an example of a student's profile. 



Position at the court* leader board, by grade*: 76 


Position at the course leader board, by merits: 11 
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Figure 3. Example of the personal profile of a student 


4. Assessing the impact of a pilot test 

We have been conducting a pilot test during year 2016, involving 15/16's second semester and 
16/17's first one. In this test, 950. students enrolled in eleven subjects of the Degree in 
Computer Engineering at the Autonomous University of Barcelona (UAB) may participate. All 
these students are part of the experience as they appear in the leaderboards and get merits, but 
are not required to participate nor receive any penalty for not doing so. At the time of writing 
this communication we do not have enough indicators to determine if the pilot has reached its 
goal. However, in order to assess the reception that this initiative is having among students 
participating, we have carried out two activities: 

• a qualitative analysis of the learning experience through surveys with the students and 
interviews with the teachers; 

• a quantitative analysis of the interaction statistics between the students and the web 
platform. 
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A total of 656 surveys were completed, the students participating in the pilot project have 
answered three questions in a five-Likert scale. The three questions referred to the student's level 
of participation in the gamification experience, the perceived usefulness and if they enjoyed 
participating on it. As in Bravo, Amante, Simo, Enache and Fernandez (2011), students have also 
expressed their opinion about a negative and a positive aspect of the experience via two open 
questions. 

In all subjects, students are divided into two profiles based on their responses: 

• Favorable students, responding to all three Likert scale questions with values greater than 
or equal to 3 (neutral element) and mention the word "motivation" at the open question 
of the positive section; and 

• Unfavorable students responding to all three questions with values less than or equal to 3 
and have mentioned "lack of time" or "no rewards" at the negative open question. 

From the eight subjects surveyed, in four of them the number of favorable and unfavorable 
students has been very similar (with acceptance rates of between 42% and 54%). At the 
remaining four, the number of unfavorable students (around 60%) has doubled the favorable 
ones (around 30%). 


Amount of authentications per day 

240 



8/2/16 19/3/16 5/5/16 


Figure 4. Amount of authentications at the internal area of TOP ENGINYERIA by day 


- 156 - 










Journal of Technology and Science Education — https://doi.org/10.3926/jotse.246 


Regarding the traffic generated on the web platform, Figure 4 shows the number of 
authentications per day taken during 15/16 second semester. As you can clearly see, there are a 
few peaks of interest during the first days that relate to the presentation of the platform to the 
students, and the first assignments of merits from the subjects with more students, then there is a 
valley that corresponds to the period of Easter holidays, and later retrieves similar values to those 
of the first half of the semester if we ignore the peaks produced the introduction of the 
experience. Needless to say, according to the usual pattern in the School of Engineering, after a 
holiday period, the same values of previous attendance and delivery of activities are never 
recovered, therefore, we consider a success that the amount of accesses is quite similar, because 
this indicates that students have not forgotten or ignored the gamification experience. 

Note that an important part of the web platform is open to the public without having to 
authenticate (any visitor can see the merits and the first 50 positions in the leaderboards), so part 
of the received visit are not reflected in this chart. Regarding the most visited pages of the web 
platform, it is surprising that the most visited one is the personal profile, with 30% more visits 
than the internal leaderboard (which lists all students). 

In addition, the fact that the public leaderboard has almost as many visits as the internal one 
indicates that the actual number of visits per day is approximately twice as shown in Figure 4. 
There are 322 of the 905 students that have accessed the web platform to view their profile, their 
merits or their position at the leaderboard. From these, only 27 have done so once. This means 
that almost 30% of students were interested enough to authenticate two or more times. There are 
eight students who were interested enough to access more than once a day (at the date of writing 
of this communication, this means 90 or more accesses), and there is a student that have 
accessed 185 times in 90 days. This data show that there is a part of the students, despite being 
very small, that feels really involved with the experience. 
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Percentage of active students, grouped by average grades 
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Figure 5. The percentage of active students decreases linearly to their average grades 


Figure 5 shows the percentage of active students (we consider a student as active if it has logged 
in at least two times) based on their average grades. It is quite remarkable to see how the 
participation of students who get better grades is very high, over 75% among students with 
average grade greater than or equal to 8. The participation of students with grades above 6 is also 
very positive (around 50%), the participation drops to 25% among students with a grade average 
around 5, and among those with a grade lesser than 4, participation is completely marginal. 
Assuming that it is impossible to motivate everyone, and that those with the worst grades are 
probably the students who attend less, spend fewer hours or take their studies a less serious, to 
provide an element of motivation that makes them do more (or make them better) to more than 
half the students averaging higher than 6 can also be considered a success. 

Finally, it is worthy to note that some of the merits has not had among the students the kind of 
reception that was expected at the moment of their design, and this has ultimately resulted in the 
introduction of small changes in the designing process by teachers. 

For example, a merit for the subject "Information and Security" which was awarded to the 
quickest students who realized an optional task, had generated a discontent among a group of 
students who wanted to have "all the merits" and felt that it was unfair to not qualify for that 
particular due to "schedule issues". As a result, the teachers of the subject changed the design of 
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the merit to grant it to all students who complete the optional task well, and have eliminated 
similar time constraints from other designed merits. 

Hidden merits, for their part, have not generated much interest in general, but in the cases where 
the teacher has provided a small hint during a lecture have managed to have an impact. In the 
subject "Advanced Internet Technologies" for the first time in four years, the web forum of the 
course was active during the Easter holidays as a result of a hidden merit awarded to the students 
that posted interesting interactions realized about a topic that was mentioned in the last lectures. 

Maybe this kind of merit might be redesigned in the coming years to increase its impact among 
students. Likewise, since many students have mentioned the word "competitiveness" when 
referring to the gamification experience, it might be a good idea to design collaborative merits 
that require pairs or groups of students to work together to achieve them. 


5. Conclusions 

In order to improve the experience, we already have made some changes during this last months, 
i.e. the inclusion of a leaderboard filter by subject. This measure has been in great demand by 
students who are enrolled to less subjects than their classmates, and felt unable to access to the 
top of the leaderboard. In addition, as one of the more criticized things has been the lack of 
rewards, we will study the possibility of getting sponsors who are willing to give gifts to the 
winners, as well as the option to give extra points to the first students of every subject 
leaderboard, in order to give some extra incentives to the students who want to participate. 

Besides, in order to try to reach as many students as possible, we will involve some subjects from 
the first semester, in order to produce an experience that lasts throughout the whole academic 
course. Overall, the participating teachers considered the experience as positive. Therefore, this 
experience will continue during the course 2016/17 with the eleven subjects of the Degree in 
Computer Engineering at the Autonomous University of Barcelona (UAB) that have participated 
this year, and it will be extended with the inclusion some other subjects. This way, we will 
continue to improve the experience and to study its impact on the student's performance. 


- 159 - 



Journal of Technology and Science Education — https://doi.org/10.3926/jotse.246 


References 

Barata, G., Gama, S., Jorge, J, & Gonsalves, D. (2013a). So Fun It Hurts - Gamifying an 
Engineering Course. Proceedings of the Foundations of Augmented Cognition: 7th International Conference. 
Las Vegas, NV, USA. https://doi.org/10.1007/978-3-642-39454-6_68 

Barata, G, Gama, S., Jorge, J., & Gonsalves, D. (2013b). Engaging Engineering Students with 
Gamification. Proceedings of the 5th International Conference on Games and l 7 irtual Worlds for Serious 
Applications (VS-GAMES), Poole, 2013, 1-8. https://doi.org/10.1109/VS-GAMES.2013.6624228 

Bartle, R. (1996). Hearts, Clubs, Diamonds, Spades: Players who suit MUDs. The Journal of 
Virtual Environments , 1(1). Retrieved from: http: / / www.mud.co.uk/richard/hcds.htm 

Bergmann, J., & Sams, A. (2012). Flip jour classroom: Pleach every student in every class every day. San 
Diego, CA: International Society for Technology in Education. 

Biggs, J. (2003). Teaching for Quality Teaming at University (2nd ed.). Milton Keynes, United 
Kingdom: Open University Press. ISBN: 0335242758. 

Bravo, E., Amante, B., Simo, P., Enache, M., & Fernandez, V. (2011). T ^ideo as a new teaching tool to 
increase student motivation. IEEE Global Engineering Education Conference (EDUCON), 
Amman, 638-642. https://doi.org/10.1109/EDUCON.2011.5773205 

Cortizo Perez, J.C., Carrero Garda, F.M., Monsalve Piqueras, B., Velasco Collado, A., Diaz del 
Dedo, L.I., & Perez Martin, J. (2011). Gamificaciony docencia: lo que la universidad tiene que aprender de 
los videojuegos. Actas de las VIII Jornadas Internacionales de Innovation Universitaria. Madrid. 
Retrieved from: http://hdl.handle.net/11268/1750 

Davis, K. (2015). Digital badges in afterschool learning: Documenting the perspectives and 
experiences of students and educators. Computers & Education, 88, 72-83. 
https://doi.org/10.1016/jxompedu.2015.04.011 

Deci, E., & Ryan, R.M. (1985). Intrinsic motivation and self-determination in human behaviour. New York: 
Plenum. 

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to 
gamefulness: Defining gamification. Proceedings of the 15th International Academic MindTrek 
Conference: Envisioning Future Media Environments, Tampere Finland. ACM, 9-15. 
https://doi.org/10.1145/2181037.2181040 


- 160 - 





Journal of Technology and Science Education — https://doi.org/10.3926/jotse.246 


Gonzalez Gonzalez, C.S., & Mora Carreno, A. (2015). Tecnicas de gamificacion aplicadas en la 
docencia de Ingenierla Informatica. Revista de Investigacion en Docencia Universitaria de la Informatica, 
8(1). ISSN 1989-1199 

Lee, J.J., & Hammer, J. (2011). Gamification in Education: What, How, Why Bother?. Academic 
Exchange Quarterly, 15(2), 1-5. 

Lopez Fernandez, D., Alarcon Cavero, P.P., Rodriguez Sanchez, M., & Casado Fuente, M.L. 
(2014). Motivacion en estudiantes de ingenierla: Un caso de estudio con teorlas e instrumentos 
para su medida y desarrollo. REDU. Revista De Docencia Universitaria, 12(4), 343-376. Retrieved 

from: http: / /red-u.net/redu/index.php / REDU / article /view/637 

O'Donovan, S., Gain, J., & Marais, P. (2013). A Case Study in the Gamification of a University- 
level Games Development Course. Proceedings of the South African Institute for Computer Sdentists and 
Information Technologists Conference (SAICSIT). South Africa, https://doi.org/10.1145/2513456.2513469 

Prieto Martin, A., Diaz Martin, D., Montserrat Sanz, J., & Reyes, E. (2014). Experiencias de 
aplicacion de estrategias de gamificacion a entornos de aprendizaje universitario. Raevista de 
Investigacion en Docencia Universitaria de la Informatica, 7(2). ISSN 1989-1199 

Smith-Robbins, S. (2011). “This Game Sucks”: How to Improve the Gamification of Education. 
EDUCAUSE Revien-, 467(1), 58-59. Retrieved from: http://er.educause.edu/ 


Published by OmniaScience ('www.omniascience.com') 

Journal of Technology and Science Education, 2017 (www.iotse.org) 

© 

EcaamaaEMD 

Article's contents are provided on an Attribution-Non Commercial 3.0 Creative commons license. Readers are 
allowed to copy, distribute and communicate article's contents, provided the author's and JOTSE journal's names are 
included. It must not be used for commercial purposes. To see the complete licence contents, please visit 
http://creativecommons.Org/licenses/by-nc/3.0/es/ 


- 161 - 










